HDD: a hypercube division-based algorithm for discretisation

This article may be used for research, teaching and private study purposes. Any substantial or systematic reproduction, redistribution , reselling , loan or sub-licensing, systematic supply or distribution in any form to anyone is expressly forbidden. The publisher does not give any warranty express or implied or make any representation that the contents will be complete or accurate or up to date. The accuracy of any instructions, formulae and drug doses should be independently verified with primary sources. The publisher shall not be liable for any loss, actions, claims, proceedings, demand or costs or damages whatsoever or howsoever caused arising directly or indirectly in connection with or arising out of the use of this material. Discretisation, as one of the basic data preparation techniques, has played an important role in data mining. This article introduces a new hypercube division-based (HDD) algorithm for supervised discretisation. The algorithm considers the distribution of both class and continuous attributes and the underlying correlation structure in the data set. It tries to find a minimal set of cut points, which divides the continuous attribute space into a finite number of hypercubes, and the objects within each hypercube belong to the same decision class. Finally, tests are performed on seven mix-mode data sets, and the C5.0 algorithm is used to generate classification rules from the discretised data. Compared with the other three well-known discretisation algorithms, the HDD algorithm can generate a better discretisation scheme, which improves the accuracy of classification and reduces the number of classification rules. 1. Introduction With the rapid development of computer and internet technologies, the amount of data and information grows exponentially. Since data mining is an extremely powerful approach to extract useful knowledge from large databases, it has become a research focus in Many data mining algorithms require that the training data contains only discrete attributes. In practice, however, a large number of attributes are of a continuous nature. In order to use these algorithms, the continuous attributes must first be discretised. This demands studies on appropriate discretisation methods. Discretisation is a process that divides the values domain of a continuous attribute into a small number of intervals, where each interval is mapped to a numerical, discrete value. After discretisation, data can be reduced and simplified. Thus, results obtained through decision trees or induction rules are usually more compact, shorter and more accurate than results derived using continuous values. Discretisation, as …